Tootfinder

IBM open sources its Granite code models for code generative tasks, trained on 116 programming languages, with models ranging in size from 3B to 34B parameters (Mike Murphy/IBM Research)
https://research.ibm.com/blog/granite-code-models-open-source

IBM’s Granite code model family is going open source
IBM is open sourcing family of Granite code models to make coding as easy as possible — for as many developers as possible.

IBM open sources its Granite code models for code generative tasks, trained on 116 programming languages, with models ranging in size from 3B to 34B parameters (Mike Murphy/IBM Research)
https://research.ibm.com/blog/granite-code-models-open-source

IBM’s Granite code model family is going open source
IBM is open sourcing family of Granite code models to make coding as easy as possible — for as many developers as possible.

SEME at SemEval-2024 Task 2: Comparing Masked and Generative Language Models on Natural Language Inference for Clinical Trials
Mathilde Aguiar, Pierre Zweigenbaum, Nona Naderi
https://arxiv.org/abs/2404.03977

SEME at SemEval-2024 Task 2: Comparing Masked and Generative Language Models on Natural Language Inference for Clinical Trials
This paper describes our submission to Task 2 of SemEval-2024: Safe Biomedical Natural Language Inference for Clinical Trials. The Multi-evidence Natural Language Inference for Clinical Trial Data (NLI4CT) consists of a Textual Entailment (TE) task focused on the evaluation of the consistency and faithfulness of Natural Language Inference (NLI) models applied to Clinical Trial Reports (CTR). We test 2 distinct approaches, one based on finetuning and ensembling Masked Language Models and the oth…

The 2nd Workshop on Recommendation with Generative Models
Wenjie Wang, Yang Zhang, Xinyu Lin, Fuli Feng, Weiwen Liu, Yong Liu, Xiangyu Zhao, Wayne Xin Zhao, Yang Song, Xiangnan He
https://arxiv.org/abs/2403.04399

The 2nd Workshop on Recommendation with Generative Models
The rise of generative models has driven significant advancements in recommender systems, leaving unique opportunities for enhancing users' personalized recommendations. This workshop serves as a platform for researchers to explore and exchange innovative concepts related to the integration of generative models into recommender systems. It primarily focuses on five key perspectives: (i) improving recommender algorithms, (ii) generating personalized content, (iii) evolving the user-system intera…

Membership Inference Attacks and Privacy in Topic Modeling
Nico Manzonelli, Wanrong Zhang, Salil Vadhan
https://arxiv.org/abs/2403.04451 https://

Membership Inference Attacks and Privacy in Topic Modeling
Recent research shows that large language models are susceptible to privacy attacks that infer aspects of the training data. However, it is unclear if simpler generative models, like topic models, share similar vulnerabilities. In this work, we propose an attack against topic models that can confidently identify members of the training data in Latent Dirichlet Allocation. Our results suggest that the privacy risks associated with generative modeling are not restricted to large neural models. Ad…

Amazon launches Bedrock Studio in public preview, a web tool to help orgs experiment with and collaborate on generative AI models and then build AI-powered apps (Kyle Wiggers/TechCrunch)
https://techcrunch.com/2024/05/07/bedr

Bedrock Studio is Amazon's attempt to simplify generative AI app development | TechCrunch
Amazon has launched a new tool, Bedrock Studio, designed to make it easier for developers to refine and launch generative AI-powered apps.

Interactive Bayesian Generative Models for Abnormality Detection in Vehicular Networks
Nobel J. William, Ali Krayani, Lucio Marcenaro, Carlo Regazzoni
https://arxiv.org/abs/2403.03583

Interactive Bayesian Generative Models for Abnormality Detection in Vehicular Networks
The following paper proposes a novel Vehicle-to-Everything (V2X) network abnormality detection scheme based on Bayesian generative models for enhanced network self-awareness functionality at the Base station (BS). In the learning phase, multi-modal data signals contrived by the vehicles' integrated and sensing module are imbued into data-driven Generalized Dynamic Bayesian network (GDBN) models. Following that, during the testing phase, an Interactive Modified Markov Jump Particle filter (IM-MJ…

The Role of Code Proficiency in the Era of Generative AI
Gregorio Robles, Christoph Treude, Jesus M. Gonzalez-Barahona, Raula Gaikovina Kula
https://arxiv.org/abs/2405.01565

The Role of Code Proficiency in the Era of Generative AI
At the current pace of technological advancements, Generative AI models, including both Large Language Models and Large Multi-modal Models, are becoming integral to the developer workspace. However, challenges emerge due to the 'black box' nature of many of these models, where the processes behind their outputs are not transparent. This position paper advocates for a 'white box' approach to these generative models, emphasizing the necessity of transparency and understanding in AI-generated code…

Ik lees weinig of niets de laatste maanden over hoe het nu gaat met het initiatief GPT-NL van onder andere TNO, SURF, NFI. Hoe staat het daarmee, hoe gaat men voldoende trainingsdata verzamelen van hoge kwaliteit, in lijn met onze regels en normen (Bigtech houdt zich daar niet aan) ? Wat wordt er eigenlijk ontwikkeld en op basis van welke modellen ?
Maar misschien heb ik nieuwsberichten gemist, zo ja, laat het me weten 🙂

GPT-NL versterkt Nederlandse autonomie, kennis en technologie in AI
Large language models zoals ChatGPT bieden veelbelovende technische mogelijkheden, maar er zijn ook zorgen. TNO werkt aan GPT-NL, een eigen nederlands taalmodel

Multi-modal perception for soft robotic interactions using generative models
Enrico Donato, Egidio Falotico, Thomas George Thuruthel
https://arxiv.org/abs/2404.04220

Multi-modal perception for soft robotic interactions using generative models
Perception is essential for the active interaction of physical agents with the external environment. The integration of multiple sensory modalities, such as touch and vision, enhances this perceptual process, creating a more comprehensive and robust understanding of the world. Such fusion is particularly useful for highly deformable bodies such as soft robots. Developing a compact, yet comprehensive state representation from multi-sensory inputs can pave the way for the development of complex c…

ProMoAI: Process Modeling with Generative AI
Humam Kourani, Alessandro Berti, Daniel Schuster, Wil M. P. van der Aalst
https://arxiv.org/abs/2403.04327 htt…

ProMoAI: Process Modeling with Generative AI
ProMoAI is a novel tool that leverages Large Language Models (LLMs) to automatically generate process models from textual descriptions, incorporating advanced prompt engineering, error handling, and code generation techniques. Beyond automating the generation of complex process models, ProMoAI also supports process model optimization. Users can interact with the tool by providing feedback on the generated model, which is then used for refining the process model. ProMoAI utilizes the capabilitie…

This https://arxiv.org/abs/2312.14385 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csDC_…

Generative AI Beyond LLMs: System Implications of Multi-Modal Generation
As the development of large-scale Generative AI models evolve beyond text (1D) generation to include image (2D) and video (3D) generation, processing spatial and temporal information presents unique challenges to quality, performance, and efficiency. We present the first work towards understanding this new system design space for multi-modal text-to-image (TTI) and text-to-video (TTV) generation models. Current model architecture designs are bifurcated into 2 categories: Diffusion- and Transfor…

A method for quantifying the generalization capabilities of generative models for solving Ising models
Qunlong Ma, Zhi Ma, Ming Gao
https://arxiv.org/abs/2405.03435

A method for quantifying the generalization capabilities of generative models for solving Ising models
For Ising models with complex energy landscapes, whether the ground state can be found by neural networks depends heavily on the Hamming distance between the training datasets and the ground state. Despite the fact that various recently proposed generative models have shown good performance in solving Ising models, there is no adequate discussion on how to quantify their generalization capabilities. Here we design a Hamming distance regularizer in the framework of a class of generative models, …

Generative Explanations for Program Synthesizers
Amirmohammad Nazari, Souti Chattopadhyay, Swabha Swayamdipta, Mukund Raghothaman
https://arxiv.org/abs/2403.03429

Generative Explanations for Program Synthesizers
Despite great advances in program synthesis techniques, they remain algorithmic black boxes. Although they guarantee that when synthesis is successful, the implementation satisfies the specification, they provide no additional information regarding how the implementation works or the manner in which the specification is realized. One possibility to answer these questions is to use large language models (LLMs) to construct human-readable explanations. Unfortunately, experiments reveal that LLMs …

Exploring Text-based Realistic Building Facades Editing Applicaiton
Jing Wang, Xin Zhang
https://arxiv.org/abs/2405.02967 https://arx…

Exploring Text-based Realistic Building Facades Editing Applicaiton
This paper explores the utilization of diffusion models and textual guidance for achieving localized editing of building facades, addressing the escalating demand for sophisticated editing methodologies in architectural design and urban planning. Leveraging the robust generative capabilities of diffusion models, this study presents a promising avenue for realistically synthesizing and modifying architectural facades. Through iterative diffusion and text descriptions, these models adeptly captur…

This https://arxiv.org/abs/2305.18231 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

High-Fidelity Image Compression with Score-based Generative Models
Despite the tremendous success of diffusion generative models in text-to-image generation, replicating this success in the domain of image compression has proven difficult. In this paper, we demonstrate that diffusion can significantly improve perceptual quality at a given bit-rate, outperforming state-of-the-art approaches PO-ELIC and HiFiC as measured by FID score. This is achieved using a simple but theoretically motivated two-stage approach combining an autoencoder targeting MSE followed by…

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models
Bowen Zhang, Kehua Chang, Chunping Li
https://arxiv.org/abs/2404.03921 ht…

Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models
Sentence Embedding stands as a fundamental task within the realm of Natural Language Processing, finding extensive application in search engines, expert systems, and question-and-answer platforms. With the continuous evolution of large language models such as LLaMA and Mistral, research on sentence embedding has recently achieved notable breakthroughs. However, these advancements mainly pertain to fine-tuning scenarios, leaving explorations into computationally efficient direct inference method…

Interesting paper, for those of us calibrating the threat from Large Language models https://www.nature.com/articles/s41598-024-53303-w

The current state of artificial intelligence generative language models is more creative than humans on divergent thinking tasks - Scientific Reports
The emergence of publicly accessible artificial intelligence (AI) large language models such as ChatGPT has given rise to global conversations on the implications of AI capabilities. Emergent research on AI has challenged the assumption that creative potential is a uniquely human trait thus, there seems to be a disconnect between human perception versus what AI is objectively capable of creating. Here, we aimed to assess the creative potential of humans in comparison to AI. In the present study…

Multi-Agent RL-Based Industrial AIGC Service Offloading over Wireless Edge Networks
Siyuan Li, Xi Lin, Hansong Xu, Kun Hua, Xiaomin Jin, Gaolei Li, Jianhua Li
https://arxiv.org/abs/2405.02972

Multi-Agent RL-Based Industrial AIGC Service Offloading over Wireless Edge Networks
Currently, the generative model has garnered considerable attention due to its application in addressing the challenge of scarcity of abnormal samples in the industrial Internet of Things (IoT). However, challenges persist regarding the edge deployment of generative models and the optimization of joint edge AI-generated content (AIGC) tasks. In this paper, we focus on the edge optimization of AIGC task execution and propose GMEL, a generative model-driven industrial AIGC collaborative edge lear…

This https://arxiv.org/abs/2310.01690 has been replaced.
initial toot: https://mastoxiv.page/@arXi…

Forecasting Tropical Cyclones with Cascaded Diffusion Models
As tropical cyclones become more intense due to climate change, the rise of Al-based modelling provides a more affordable and accessible approach compared to traditional methods based on mathematical models. This work leverages generative diffusion models to forecast cyclone trajectories and precipitation patterns by integrating satellite imaging, remote sensing, and atmospheric data. It employs a cascaded approach that incorporates three main tasks: forecasting, super-resolution, and precipita…

Diffusion-based Generative Prior for Low-Complexity MIMO Channel Estimation
Benedikt Fesl, Michael Baur, Florian Strasser, Michael Joham, Wolfgang Utschick
https://arxiv.org/abs/2403.03545

Diffusion-based Generative Prior for Low-Complexity MIMO Channel Estimation
This work proposes a novel channel estimator based on diffusion models (DMs), one of the currently top-rated generative models. Contrary to related works utilizing generative priors, a lightweight convolutional neural network (CNN) with positional embedding of the signal-to-noise ratio (SNR) information is designed by learning the channel distribution in the sparse angular domain. Combined with an estimation strategy that avoids stochastic resampling and truncates reverse diffusion steps that a…

AtomGPT: Atomistic Generative Pre-trained Transformer for Forward and Inverse Materials Design
Kamal Choudhary
https://arxiv.org/abs/2405.03680 https://

AtomGPT: Atomistic Generative Pre-trained Transformer for Forward and Inverse Materials Design
Large language models (LLMs) such as generative pretrained transformers (GPTs) have shown potential for various commercial applications, but their applicability for materials design remains underexplored. In this article, we introduce AtomGPT, a model specifically developed for materials design based on transformer architectures, to demonstrate the capability for both atomistic property prediction and structure generation. We show that a combination of chemical and structural text descriptions …

How LLMs Work, Explained Without Math https://blog.miguelgrinberg.com/post/how-llms-work-explained-without-math

Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design
A N M Nafiz Abeer, Sanket Jantre, Nathan M Urban, Byung-Jun Yoon
https://arxiv.org/abs/2405.00202

Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design
Deep generative models have been accelerating the inverse design process in material and drug design. Unlike their counterpart property predictors in typical molecular design frameworks, generative molecular design models have seen fewer efforts on uncertainty quantification (UQ) due to computational challenges in Bayesian inference posed by their large number of parameters. In this work, we focus on the junction-tree variational autoencoder (JT-VAE), a popular model for generative molecular de…

Exploring Text-based Realistic Building Facades Editing Applicaiton
Jing Wang, Xin Zhang
https://arxiv.org/abs/2405.02967 https://arx…

Exploring Text-based Realistic Building Facades Editing Applicaiton
This paper explores the utilization of diffusion models and textual guidance for achieving localized editing of building facades, addressing the escalating demand for sophisticated editing methodologies in architectural design and urban planning. Leveraging the robust generative capabilities of diffusion models, this study presents a promising avenue for realistically synthesizing and modifying architectural facades. Through iterative diffusion and text descriptions, these models adeptly captur…

Wiz details two now-fixed security issues on the Hugging Face AI platform that put customer data at risk, as Hugging Face partners with Wiz to improve security (Kevin Poireault/Infosecurity)
https://www.infosecurity-magazine.com/news/wiz-discovers-fla…

Wiz Discovers Flaws in GenAI Models Enabling Customer Data Theft
Wiz researchers found architecture flaws in generative AI models available on the AI hub Hugging Face

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M\"uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, Robin Rombach
https://…

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is not yet decisively established as standard practice. In this work, we improve existing noise samplin…

This https://arxiv.org/abs/2401.11389 has been replaced.
link: https://scholar.google.com/scholar?q=a

MedLM: Exploring Language Models for Medical Question Answering Systems
In the face of rapidly expanding online medical literature, automated systems for aggregating and summarizing information are becoming increasingly crucial for healthcare professionals and patients. Large Language Models (LLMs), with their advanced generative capabilities, have shown promise in various NLP tasks, and their potential in the healthcare domain, particularly for Closed-Book Generative QnA, is significant. However, the performance of these models in domain-specific tasks such as med…

This https://arxiv.org/abs/2401.03968 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_qbi…

scDiffusion: conditional generation of high-quality single-cell data using diffusion model
Single-cell RNA sequencing (scRNA-seq) data are important for studying the laws of life at single-cell level. However, it is still challenging to obtain enough high-quality scRNA-seq data. To mitigate the limited availability of data, generative models have been proposed to computationally generate synthetic scRNA-seq data. Nevertheless, the data generated with current models are not very realistic yet, especially when we need to generate data with controlled conditions. In the meantime, the Di…

"Huge generative #AI models like #ChatGPT [are] great at things I never would have expected an #LLM-based program to be good at, like writing certain kinds of computer programs, summarizing and editing text, and a whol…

Generative AI is a hammer and no one knows what is and isn’t a nail
This analogy is going to seem a bit tortured but bear with me. Imagine a world without hammers. You’re driving nails into the wall with your bare hands to hang up your paintings. You’re kicking…

CALRec: Contrastive Alignment of Generative LLMs For Sequential Recommendation
Yaoyiran Li, Xiang Zhai, Moustafa Alzantot, Keyi Yu, Ivan Vuli\'c, Anna Korhonen, Mohamed Hammad
https://arxiv.org/abs/2405.02429

CALRec: Contrastive Alignment of Generative LLMs For Sequential Recommendation
Traditional recommender systems such as matrix factorization methods rely on learning a shared dense embedding space to represent both items and user preferences. Sequence models such as RNN, GRUs, and, recently, Transformers have also excelled in the task of sequential recommendation. This task requires understanding the sequential structure present in users' historical interactions to predict the next item they may like. Building upon the success of Large Language Models (LLMs) in a variety o…

Generative AI in Cybersecurity
Shivani Metta, Isaac Chang, Jack Parker, Michael P. Roman, Arturo F. Ehuan
https://arxiv.org/abs/2405.01674 https://<…

Generative AI in Cybersecurity
The dawn of Generative Artificial Intelligence (GAI), characterized by advanced models such as Generative Pre-trained Transformers (GPT) and other Large Language Models (LLMs), has been pivotal in reshaping the field of data analysis, pattern recognition, and decision-making processes. This surge in GAI technology has ushered in not only innovative opportunities for data processing and automation but has also introduced significant cybersecurity challenges. As GAI rapidly progresses, it outst…

MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant
Chenlu Zhan, Yu Lin, Gaoang Wang, Hongwei Wang, Jian Wu
https://arxiv.org/abs/2403.04290

MedM2G: Unifying Medical Multi-Modal Generation via Cross-Guided Diffusion with Visual Invariant
Medical generative models, acknowledged for their high-quality sample generation ability, have accelerated the fast growth of medical applications. However, recent works concentrate on separate medical generation models for distinct medical tasks and are restricted to inadequate medical multi-modal knowledge, constraining medical comprehensive diagnosis. In this paper, we propose MedM2G, a Medical Multi-Modal Generative framework, with the key innovation to align, extract, and generate medical …

Whodunit: Classifying Code as Human Authored or GPT-4 Generated -- A case study on CodeChef problems
Oseremen Joy Idialu, Noble Saji Mathews, Rungroj Maipradit, Joanne M. Atlee, Mei Nagappan
https://arxiv.org/abs/2403.04013

Whodunit: Classifying Code as Human Authored or GPT-4 Generated -- A case study on CodeChef problems
Artificial intelligence (AI) assistants such as GitHub Copilot and ChatGPT, built on large language models like GPT-4, are revolutionizing how programming tasks are performed, raising questions about whether code is authored by generative AI models. Such questions are of particular interest to educators, who worry that these tools enable a new form of academic dishonesty, in which students submit AI generated code as their own work. Our research explores the viability of using code stylometry a…

"Training Data for the Price of a Sandwich – Common Crawl’s Impact on Generative AI "
https://foundation.mozilla.org/de/research/library/generative-ai-training-data/common-crawl/

Training Data for the Price of a Sandwich: Common Crawl’s Impact on Generative AI
Mozilla research finds that Common Crawl's outsized role in the generative AI boom has improved transparency and competition, but is also contributing to biased and opaque generative AI models.

MMGER: Multi-modal and Multi-granularity Generative Error Correction with LLM for Joint Accent and Speech Recognition
Bingshen Mu, Yangze Li, Qijie Shao, Kun Wei, Xucheng Wan, Naijun Zheng, Huan Zhou, Lei Xie
https://arxiv.org/abs/2405.03152

MMGER: Multi-modal and Multi-granularity Generative Error Correction with LLM for Joint Accent and Speech Recognition
Despite notable advancements in automatic speech recognition (ASR), performance tends to degrade when faced with adverse conditions. Generative error correction (GER) leverages the exceptional text comprehension capabilities of large language models (LLM), delivering impressive performance in ASR error correction, where N-best hypotheses provide valuable information for transcription prediction. However, GER encounters challenges such as fixed N-best hypotheses, insufficient utilization of acou…

This https://arxiv.org/abs/2405.00711 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities
In recent years, generative artificial intelligence models, represented by Large Language Models (LLMs) and Diffusion Models (DMs), have revolutionized content production methods. These artificial intelligence-generated content (AIGC) have become deeply embedded in various aspects of daily life and work, spanning texts, images, videos, and audio. The authenticity of AI-generated content is progressively enhancing, approaching human-level creative standards. However, these technologies have also…

This https://arxiv.org/abs/2404.02029 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCE_…

Enhancing Portfolio Optimization with Transformer-GAN Integration: A Novel Approach in the Black-Litterman Framework
This study presents an innovative approach to portfolio optimization by integrating Transformer models with Generative Adversarial Networks (GANs) within the Black-Litterman (BL) framework. Capitalizing on Transformers' ability to discern long-range dependencies and GANs' proficiency in generating accurate predictive models, our method enhances the generation of refined predictive views for BL portfolio allocations. This fusion of our model with BL's structured method for merging objective view…

Creation of Novel Soft Robot Designs using Generative AI
Wee Kiat Chan, PengWei Wang, Raye Chen-Hua Yeow
https://arxiv.org/abs/2405.01824 https://

Creation of Novel Soft Robot Designs using Generative AI
Soft robotics has emerged as a promising field with the potential to revolutionize industries such as healthcare and manufacturing. However, designing effective soft robots presents challenges, particularly in managing the complex interplay of material properties, structural design, and control strategies. Traditional design methods are often time-consuming and may not yield optimal designs. In this paper, we explore the use of generative AI to create 3D models of soft actuators. We create a da…

Unexpected potential application of generative-AI technology; looks attractive in theory: https://www.rfc-editor.org/rfc/rfc9564.html

Solar synthetic imaging: Introducing denoising diffusion probabilistic models on SDO/AIA data
Francesco P. Ramunno, S. Hackstein, V. Kinakh, M. Drozdova, G. Quetant, A. Csillaghy, S. Voloshynovskiy
https://arxiv.org/abs/2404.02552

Solar synthetic imaging: Introducing denoising diffusion probabilistic models on SDO/AIA data
Given the rarity of significant solar flares compared to smaller ones, training effective machine learning models for solar activity forecasting is challenging due to insufficient data. This study proposes using generative deep learning models, specifically a Denoising Diffusion Probabilistic Model (DDPM), to create synthetic images of solar phenomena, including flares of varying intensities. By employing a dataset from the AIA instrument aboard the SDO spacecraft, focusing on the 171 Å band t…

How LLMs Work, Explained Without Math https://blog.miguelgrinberg.com/post/how-llms-work-explained-without-math

Can Nuanced Language Lead to More Actionable Insights? Exploring the Role of Generative AI in Analytical Narrative Structure
Vidya Setlur, Larry Birnbaum
https://arxiv.org/abs/2405.02763

Can Nuanced Language Lead to More Actionable Insights? Exploring the Role of Generative AI in Analytical Narrative Structure
Relevant language describing trends in data can be useful for generating summaries to help with readers' takeaways. However, the language employed in these often template-generated summaries tends to be simple, ranging from describing simple statistical information (e.g., extrema and trends) without additional context and richer language to provide actionable insights. Recent advances in Large Language Models (LLMs) have shown promising capabilities in capturing subtle nuances in language when …

Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design
A N M Nafiz Abeer, Sanket Jantre, Nathan M Urban, Byung-Jun Yoon
https://arxiv.org/abs/2405.00202

Leveraging Active Subspaces to Capture Epistemic Model Uncertainty in Deep Generative Models for Molecular Design
Deep generative models have been accelerating the inverse design process in material and drug design. Unlike their counterpart property predictors in typical molecular design frameworks, generative molecular design models have seen fewer efforts on uncertainty quantification (UQ) due to computational challenges in Bayesian inference posed by their large number of parameters. In this work, we focus on the junction-tree variational autoencoder (JT-VAE), a popular model for generative molecular de…

This https://arxiv.org/abs/2403.00867 has been replaced.
link: https://scholar.google.com/scholar?q=a

Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes
Large Language Models (LLMs) are becoming a prominent generative AI tool, where the user enters a query and the LLM generates an answer. To reduce harm and misuse, efforts have been made to align these LLMs to human values using advanced training techniques such as Reinforcement Learning from Human Feedback (RLHF). However, recent studies have highlighted the vulnerability of LLMs to adversarial jailbreak attempts aiming at subverting the embedded safety guardrails. To address this challenge, t…

KC-GenRe: A Knowledge-constrained Generative Re-ranking Method Based on Large Language Models for Knowledge Graph Completion
Yilin Wang, Minghao Hu, Zhen Huang, Dongsheng Li, Dong Yang, Xicheng Lu
https://arxiv.org/abs/2403.17532

KC-GenRe: A Knowledge-constrained Generative Re-ranking Method Based on Large Language Models for Knowledge Graph Completion
The goal of knowledge graph completion (KGC) is to predict missing facts among entities. Previous methods for KGC re-ranking are mostly built on non-generative language models to obtain the probability of each candidate. Recently, generative large language models (LLMs) have shown outstanding performance on several tasks such as information extraction and dialog systems. Leveraging them for KGC re-ranking is beneficial for leveraging the extensive pre-trained knowledge and powerful generative c…

This https://arxiv.org/abs/2404.05468 has been replaced.
link: https://scholar.google.com/scholar?q=a

Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI
The reconstruction of images observed by subjects from fMRI data collected during visual stimuli has made strong progress in the past decade, thanks to the availability of extensive fMRI datasets and advancements in generative models for image generation. However, the application of visual reconstruction has remained limited. Reconstructing visual imagination presents a greater challenge, with potentially revolutionary applications ranging from aiding individuals with disabilities to verifying …

Generative Software Engineering
Yuan Huang, Yinan Chen, Xiangping Chen, Junqi Chen, Rui Peng, Zhicao Tang, Jinbo Huang, Furen Xu, Zibin Zheng
https://arxiv.org/abs/2403.02583

Generative Software Engineering
The rapid development of deep learning techniques, improved computational power, and the availability of vast training data have led to significant advancements in pre-trained models and large language models (LLMs). Pre-trained models based on architectures such as BERT and Transformer, as well as LLMs like ChatGPT, have demonstrated remarkable language capabilities and found applications in Software engineering. Software engineering tasks can be divided into many categories, among which gener…

Just wow...amazing website/visualization about LAION-5B , a large dataset a lot of generative AI models are trained on.
#AI

GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation
Kaustubh Dhole, Eugene Agichtein
https://arxiv.org/abs/2404.03746 https:…

GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation
Query Reformulation(QR) is a set of techniques used to transform a user's original search query to a text that better aligns with the user's intent and improves their search experience. Recently, zero-shot QR has been shown to be a promising approach due to its ability to exploit knowledge inherent in large language models. By taking inspiration from the success of ensemble prompting strategies which have benefited many tasks, we investigate if they can help improve query reformulation. In this…

Mitigating analytical variability in fMRI results with style transfer
Elodie Germani (EMPENN, LACODAM), Elisa Fromont (LACODAM), Camille Maumet (EMPENN)
https://arxiv.org/abs/2404.03703

Mitigating analytical variability in fMRI results with style transfer
We propose a novel approach to improve the reproducibility of neuroimaging results by converting statistic maps across different functional MRI pipelines. We make the assumption that pipelines can be considered as a style component of data and propose to use different generative models, among which, Diffusion Models (DM) to convert data between pipelines. We design a new DM-based unsupervised multi-domain image-to-image transition framework and constrain the generation of 3D fMRI statistic maps…

Can Nuanced Language Lead to More Actionable Insights? Exploring the Role of Generative AI in Analytical Narrative Structure
Vidya Setlur, Larry Birnbaum
https://arxiv.org/abs/2405.02763

Can Nuanced Language Lead to More Actionable Insights? Exploring the Role of Generative AI in Analytical Narrative Structure
Relevant language describing trends in data can be useful for generating summaries to help with readers' takeaways. However, the language employed in these often template-generated summaries tends to be simple, ranging from describing simple statistical information (e.g., extrema and trends) without additional context and richer language to provide actionable insights. Recent advances in Large Language Models (LLMs) have shown promising capabilities in capturing subtle nuances in language when …

The point of writing is not to generate a written document
It's to communicate an idea. In business writing it's usually to communicate a _persuasive_ idea
The people @… referred to here,who dislike reading and writing and would rather a machine generated that writing for them — they don't get this bit.

Indian watchdog MeitY issues an advisory asking developers to seek "explicit permission" before deploying any AI models or AI-backed algorithms (The Economic Times)
https://economictimes.indiatimes…

MeitY approval must for companies to roll out AI, generative AI models
In its advisory sent late on Friday, the IT ministry has also said that such platforms will have to explicitly seek permission from the government to operate in India and must offer disclaimers and disclosures of their platforms being under testing.

Physics-informed generative neural networks for RF propagation prediction with application to indoor body perception
Federica Fieramosca, Vittorio Rampa, Michele D'Amico, Stefano Savazzi
https://arxiv.org/abs/2405.02131

Physics-informed generative neural networks for RF propagation prediction with application to indoor body perception
Electromagnetic (EM) body models designed to predict Radio-Frequency (RF) propagation are time-consuming methods which prevent their adoption in strict real-time computational imaging problems, such as human body localization and sensing. Physics-informed Generative Neural Network (GNN) models have been recently proposed to reproduce EM effects, namely to simulate or reconstruct missing data or samples by incorporating relevant EM principles and constraints. The paper discusses a Variational Au…

Fast, Scale-Adaptive, and Uncertainty-Aware Downscaling of Earth System Model Fields with Generative Foundation Models
Philipp Hess, Michael Aich, Baoxiang Pan, Niklas Boers
https://arxiv.org/abs/2403.02774

Fast, Scale-Adaptive, and Uncertainty-Aware Downscaling of Earth System Model Fields with Generative Foundation Models
Accurate and high-resolution Earth system model (ESM) simulations are essential to assess the ecological and socio-economic impacts of anthropogenic climate change, but are computationally too expensive. Recent machine learning approaches have shown promising results in downscaling ESM simulations, outperforming state-of-the-art statistical approaches. However, existing methods require computationally costly retraining for each ESM and extrapolate poorly to climates unseen during training. We a…

Sketch-to-Architecture: Generative AI-aided Architectural Design
Pengzhi Li, Baijuan Li, Zhiheng Li
https://arxiv.org/abs/2403.20186 https://

Sketch-to-Architecture: Generative AI-aided Architectural Design
Recently, the development of large-scale models has paved the way for various interdisciplinary research, including architecture. By using generative AI, we present a novel workflow that utilizes AI models to generate conceptual floorplans and 3D models from simple sketches, enabling rapid ideation and controlled generation of architectural renderings based on textual descriptions. Our work demonstrates the potential of generative AI in the architectural design process, pointing towards a new d…

3D Diffusion Policy
Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, Huazhe Xu
https://arxiv.org/abs/2403.03954 https://…

3D Diffusion Policy
Imitation learning provides an efficient way to teach robots dexterous skills; however, learning complex skills robustly and generalizablely usually consumes large amounts of human demonstrations. To tackle this challenging problem, we present 3D Diffusion Policy (DP3), a novel visual imitation learning approach that incorporates the power of 3D visual representations into diffusion policies, a class of conditional action generative models. The core design of DP3 is the utilization of a compact…

Indian watchdog MeitY issues an advisory asking developers to seek "explicit permission" before deploying any AI models or AI-backed algorithms (The Economic Times)
https://economictimes.indiatimes…

MeitY approval must for companies to roll out AI, generative AI models
In its advisory sent late on Friday, the IT ministry has also said that such platforms will have to explicitly seek permission from the government to operate in India and must offer disclaimers and disclosures of their platforms being under testing.

Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics
Rucha Deshpande, Varun A. Kelkar, Dimitrios Gotsis, Prabhat Kc, Rongping Zeng, Kyle J. Myers, Frank J. Brooks, Mark A. Anastasio
https://arxiv.org/abs/2405.01822

Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics
The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. The goal of this challenge was to promote the development of deep generative models (DGMs) for medical imaging and to emphasize the need for their domain-relevant assessment via the analysis of relevant image statistics. As part of this Grand Challenge, a training dataset was developed based on 3D anthropomorphic breast phantoms from the VICTRE vir…

This https://arxiv.org/abs/2311.01616 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_ees…

Adapting Frechet Audio Distance for Generative Music Evaluation
The growing popularity of generative music models underlines the need for perceptually relevant, objective music quality metrics. The Frechet Audio Distance (FAD) is commonly used for this purpose even though its correlation with perceptual quality is understudied. We show that FAD performance may be hampered by sample size bias, poor choice of audio embeddings, or the use of biased or low-quality reference sets. We propose reducing sample size bias by extrapolating scores towards an infinite s…

Generative models struggle with kirigami metamaterials
Gerrit Felsch, Viacheslav Slesarenko
https://arxiv.org/abs/2402.19196 https://…

Generative models struggle with kirigami metamaterials
Generative machine learning models have shown notable success in identifying architectures for metamaterials - materials whose behavior is determined primarily by their internal organization - that match specific target properties. By examining kirigami metamaterials, in which dependencies between cuts yield complex design restrictions, we demonstrate that this perceived success in the employment of generative models for metamaterials might be akin to survivorship bias. We assess the performanc…

Large Language Models and Video Games: A Preliminary Scoping Review
Penny Sweetser
https://arxiv.org/abs/2403.02613 https://arxiv.org…

Large Language Models and Video Games: A Preliminary Scoping Review
Large language models (LLMs) hold interesting potential for the design, development, and research of video games. Building on the decades of prior research on generative AI in games, many researchers have sped to investigate the power and potential of LLMs for games. Given the recent spike in LLM-related research in games, there is already a wealth of relevant research to survey. In order to capture a snapshot of the state of LLM research in games, and to help lay the foundation for future work…

Behavior Generation with Latent Actions
Seungjae Lee, Yibin Wang, Haritheja Etukuru, H. Jin Kim, Nur Muhammad Mahi Shafiullah, Lerrel Pinto
https://arxiv.org/abs/2403.03181

Behavior Generation with Latent Actions
Generative modeling of complex behaviors from labeled datasets has been a longstanding problem in decision making. Unlike language or image generation, decision making requires modeling actions - continuous-valued vectors that are multimodal in their distribution, potentially drawn from uncurated sources, where generation errors can compound in sequential prediction. A recent class of models called Behavior Transformers (BeT) addresses this by discretizing actions using k-means clustering to ca…

PANGeA: Procedural Artificial Narrative using Generative AI for Turn-Based Video Games
Steph Buongiorno, Lawrence Jake Klinkert, Tanishq Chawla, Zixin Zhuang, Corey Clark
https://arxiv.org/abs/2404.19721

PANGeA: Procedural Artificial Narrative using Generative AI for Turn-Based Video Games
This research introduces Procedural Artificial Narrative using Generative AI (PANGeA), a structured approach for leveraging large language models (LLMs), guided by a game designer's high-level criteria, to generate narrative content for turn-based role-playing video games (RPGs). Distinct from prior applications of LLMs used for video game design, PANGeA innovates by not only generating game level data (which includes, but is not limited to, setting, key items, and non-playable characters (NPCs…

Vector Quantization for Recommender Systems: A Review and Outlook
Qijiong Liu, Xiaoyu Dong, Jiaren Xiao, Nuo Chen, Hengchang Hu, Jieming Zhu, Chenxu Zhu, Tetsuya Sakai, Xiao-Ming Wu
https://arxiv.org/abs/2405.03110

Vector Quantization for Recommender Systems: A Review and Outlook
Vector quantization, renowned for its unparalleled feature compression capabilities, has been a prominent topic in signal processing and machine learning research for several decades and remains widely utilized today. With the emergence of large models and generative AI, vector quantization has gained popularity in recommender systems, establishing itself as a preferred solution. This paper starts with a comprehensive review of vector quantization techniques. It then explores systematic taxonom…

This https://arxiv.org/abs/2403.02583 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csSE_…

Generative Software Engineering
The rapid development of deep learning techniques, improved computational power, and the availability of vast training data have led to significant advancements in pre-trained models and large language models (LLMs). Pre-trained models based on architectures such as BERT and Transformer, as well as LLMs like ChatGPT, have demonstrated remarkable language capabilities and found applications in Software engineering. Software engineering tasks can be divided into many categories, among which gener…

Synthesizing study-specific controls using generative models on open access datasets for harmonized multi-study analyses
Shruti P. Gadewar, Alyssa H. Zhu, Iyad Ba Gari, Sunanda Somu, Sophia I. Thomopoulos, Paul M. Thompson, Talia M. Nir, Neda Jahanshad
https://arxiv.org/abs/2403.00093

Synthesizing study-specific controls using generative models on open access datasets for harmonized multi-study analyses
Neuroimaging consortia can enhance reliability and generalizability of findings by pooling data across studies to achieve larger sample sizes. To adjust for site and MRI protocol effects, imaging datasets are often harmonized based on healthy controls. When data from a control group were not collected, statistical harmonization options are limited as patient characteristics and acquisition-related variables may be confounded. Here, in a multi-study neuroimaging analysis of Alzheimer's patients …

Functional Imaging Constrained Diffusion for Brain PET Synthesis from Structural MRI
Minhui Yu, Mengqi Wu, Ling Yue, Andrea Bozoki, Mingxia Liu
https://arxiv.org/abs/2405.02504

Functional Imaging Constrained Diffusion for Brain PET Synthesis from Structural MRI
Magnetic resonance imaging (MRI) and positron emission tomography (PET) are increasingly used in multimodal analysis of neurodegenerative disorders. While MRI is broadly utilized in clinical settings, PET is less accessible. Many studies have attempted to use deep generative models to synthesize PET from MRI scans. However, they often suffer from unstable training and inadequately preserve brain functional information conveyed by PET. To this end, we propose a functional imaging constrained dif…

Chinese city governments are offering "computing vouchers", worth $140K to $280K, to AI startups, to help create a level playing field with China's tech giants (Financial Times)
https://t.co/C0ihUB3D8K

"If it does turn out to be anything like human understanding, it will probably not be based on LLMs.
After all, LLMs learn in the opposite direction from humans. LLMs start out learning language and attempt to abstract concepts. Human babies learn concepts first, and only later acquire the language to describe them."

Why large language models aren’t headed toward humanlike understanding
Unlike people, today's generative AI isn’t good at learning concepts that it can apply to new situations.

Semi-Supervised Learning for Deep Causal Generative Models
Yasin Ibrahim, Hermione Warr, Konstantinos Kamnitsas
https://arxiv.org/abs/2403.18717 https://…

Semi-Supervised Learning for Deep Causal Generative Models
Developing models that can answer questions of the form "How would $x$ change if $y$ had been $z$?" is fundamental for advancing medical image analysis. Training causal generative models that address such counterfactual questions, though, currently requires that all relevant variables have been observed and that corresponding labels are available in training data. However, clinical data may not have complete records for all patients and state of the art causal generative models are unable to ta…

This https://arxiv.org/abs/2312.00057 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

VA3: Virtually Assured Amplification Attack on Probabilistic Copyright Protection for Text-to-Image Generative Models
The booming use of text-to-image generative models has raised concerns about their high risk of producing copyright-infringing content. While probabilistic copyright protection methods provide a probabilistic guarantee against such infringement, in this paper, we introduce Virtually Assured Amplification Attack (VA3), a novel online attack framework that exposes the vulnerabilities of these protection mechanisms. The proposed framework significantly amplifies the probability of generating infri…

This https://arxiv.org/abs/2402.12531 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCV_…

Improving Deep Generative Models on Many-To-One Image-to-Image Translation
Deep generative models have been applied to multiple applications in image- to-image translation. Generative Adversarial Networks and Diffusion Models have presented impressive results, setting new state-of-the-art results on these tasks. Most methods have symmetric setups across the different domains in a dataset. These methods assume that all domains have either multiple modalities or only one modality. However, there are many datasets that have a many-to-one relationship between two domains.…

A Domain Translation Framework with an Adversarial Denoising Diffusion Model to Generate Synthetic Datasets of Echocardiography Images
Cristiana Tiago, Sten Roar Snare, Jurica Sprem, Kristin McLeod
https://arxiv.org/abs/2403.04612

A Domain Translation Framework with an Adversarial Denoising Diffusion Model to Generate Synthetic Datasets of Echocardiography Images
Currently, medical image domain translation operations show a high demand from researchers and clinicians. Amongst other capabilities, this task allows the generation of new medical images with sufficiently high image quality, making them clinically relevant. Deep Learning (DL) architectures, most specifically deep generative models, are widely used to generate and translate images from one domain to another. The proposed framework relies on an adversarial Denoising Diffusion Model (DDM) to syn…

This https://arxiv.org/abs/2405.00208 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

A Primer on the Inner Workings of Transformer-based Language Models
The rapid progress of research aimed at interpreting the inner workings of advanced language models has highlighted a need for contextualizing the insights gained from years of work in this area. This primer provides a concise technical introduction to the current techniques used to interpret the inner workings of Transformer-based language models, focusing on the generative decoder-only architecture. We conclude by presenting a comprehensive overview of the known internal mechanisms implemente…

OpenAI expands its Custom Model training program with "assisted fine-tuning", letting organizations set up data training pipelines, evaluation systems, and more (Kyle Wiggers/TechCrunch)
https://techcrunch.com/2024/04/04/openai-expands…

OpenAI expands its custom model training program | TechCrunch
OpenAI is expanding a program, Custom Model, to help enterprise customers develop tailored generative AI models using its technology for specific use

This https://arxiv.org/abs/2306.02546 has been replaced.
link: https://scholar.google.com/scholar?q=a

Leveraging Generative Models to Recover Variable Names from Stripped Binary
Decompilation aims to recover the source code form of a binary executable. It has many security applications such as malware analysis, vulnerability detection and code hardening. A prominent challenge in decompilation is to recover variable names. We propose a novel technique that leverages the strengths of generative models while suppressing potential hallucinations and overcoming the input token limitation. We build a prototype, GenNm, from a pre-trained generative model Code-Llama. We fine-t…

Anatomy-Guided Surface Diffusion Model for Alzheimer's Disease Normative Modeling
Jianwei Zhang, Yonggang Shi
https://arxiv.org/abs/2403.04531 https://…

Anatomy-Guided Surface Diffusion Model for Alzheimer's Disease Normative Modeling
Normative modeling has emerged as a pivotal approach for characterizing heterogeneity and individual variance in neurodegenerative diseases, notably Alzheimer's disease(AD). One of the challenges of cortical normative modeling is the anatomical structure mismatch due to folding pattern variability. Traditionally, registration is applied to address this issue and recently many studies have utilized deep generative models to generate anatomically align samples for analyzing disease progression; h…

This https://arxiv.org/abs/2311.04694 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csIR_…

Evaluating Generative Ad Hoc Information Retrieval
Recent advances in large language models have enabled the development of viable generative retrieval systems. Instead of a traditional document ranking, many generative retrieval systems directly return a grounded generated text as an answer to an information need expressed as a query or question. Quantifying the utility of the textual responses is essential for appropriately evaluating such generative ad hoc retrieval. Yet, the established evaluation methodology for ranking-based retrieval is …

This https://arxiv.org/abs/2312.00057 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCR_…

VA3: Virtually Assured Amplification Attack on Probabilistic Copyright Protection for Text-to-Image Generative Models
The booming use of text-to-image generative models has raised concerns about their high risk of producing copyright-infringing content. While probabilistic copyright protection methods provide a probabilistic guarantee against such infringement, in this paper, we introduce Virtually Assured Amplification Attack (VA3), a novel online attack framework that exposes the vulnerabilities of these protection mechanisms. The proposed framework significantly amplifies the probability of generating infri…

Adobe announces Custom Models, to let businesses customize Firefly models, and Firefly Services, a set of 20 generative and creative APIs, tools, and services (Frederic Lardinois/TechCrunch)
https://techcrunch.com/2024/03/26/adob…

Adobe's Firefly Services makes over 20 new generative and creative APIs available to developers | TechCrunch
Adobe today announced Firefly Services, a set of over 20 new generative and creative APIs, tools and services. Firefly Services makes some of the

Disentangling representations of retinal images with generative models
Sarah M\"uller, Lisa M. Koch, Hendrik P. A. Lensch, Philipp Berens
https://arxiv.org/abs/2402.19186

Disentangling representations of retinal images with generative models
Retinal fundus images play a crucial role in the early detection of eye diseases and, using deep learning approaches, recent studies have even demonstrated their potential for detecting cardiovascular risk factors and neurological disorders. However, the impact of technical factors on these images can pose challenges for reliable AI applications in ophthalmology. For example, large fundus cohorts are often confounded by factors like camera type, image quality or illumination level, bearing the …

Synthesizing EEG Signals from Event-Related Potential Paradigms with Conditional Diffusion Models
Guido Klein, Pierre Guetschel, Gianluigi Silvestri, Michael Tangermann
https://arxiv.org/abs/2403.18486

Synthesizing EEG Signals from Event-Related Potential Paradigms with Conditional Diffusion Models
Data scarcity in the brain-computer interface field can be alleviated through the use of generative models, specifically diffusion models. While diffusion models have previously been successfully applied to electroencephalogram (EEG) data, existing models lack flexibility w.r.t.~sampling or require alternative representations of the EEG data. To overcome these limitations, we introduce a novel approach to conditional diffusion models that utilizes classifier-free guidance to directly generate s…

Better & Faster Large Language Models via Multi-token Prediction
Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozi\`ere, David Lopez-Paz, Gabriel Synnaeve
https://arxiv.org/abs/2404.19737 https://arxiv.org/pdf/2404.19737
arXiv:2404.19737v1 Announce Type: new
Abstract: Large language models such as GPT and Llama are trained with a next-token prediction loss. In this work, we suggest that training language models to predict multiple future tokens at once results in higher sample efficiency. More specifically, at each position in the training corpus, we ask the model to predict the following n tokens using n independent output heads, operating on top of a shared model trunk. Considering multi-token prediction as an auxiliary training task, we measure improved downstream capabilities with no overhead in training time for both code and natural language models. The method is increasingly useful for larger model sizes, and keeps its appeal when training for multiple epochs. Gains are especially pronounced on generative benchmarks like coding, where our models consistently outperform strong baselines by several percentage points. Our 13B parameter models solves 12 % more problems on HumanEval and 17 % more on MBPP than comparable next-token models. Experiments on small algorithmic tasks demonstrate that multi-token prediction is favorable for the development of induction heads and algorithmic reasoning capabilities. As an additional benefit, models trained with 4-token prediction are up to 3 times faster at inference, even with large batch sizes.

This https://arxiv.org/abs/2310.13828 has been replaced.
link: https://scholar.google.com/scholar?q=a

Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Data poisoning attacks manipulate training data to introduce unexpected behaviors into machine learning models at training time. For text-to-image generative models with massive training datasets, current understanding of poisoning attacks suggests that a successful attack would require injecting millions of poison samples into their training pipeline. In this paper, we show that poisoning attacks can be successful on generative models. We observe that training data per concept can be quite lim…

This https://arxiv.org/abs/2403.07721 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for EEG-based visual reconstruction. In this study, we present an EEG-based visual reconstruction framew…

VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
Junlin Han, Filippos Kokkinos, Philip Torr
https://arxiv.org/abs/2403.12034 h…

VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models
This paper presents a novel paradigm for building scalable 3D generative models utilizing pre-trained video diffusion models. The primary obstacle in developing foundation 3D generative models is the limited availability of 3D data. Unlike images, texts, or videos, 3D data are not readily accessible and are difficult to acquire. This results in a significant disparity in scale compared to the vast quantities of other types of data. To address this issue, we propose using a video diffusion model…

This https://arxiv.org/abs/2403.07721 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
How to decode human vision through neural signals has attracted a long-standing interest in neuroscience and machine learning. Modern contrastive learning and generative models improved the performance of fMRI-based visual decoding and reconstruction. However, the high cost and low temporal resolution of fMRI limit their applications in brain-computer interfaces (BCIs), prompting a high need for EEG-based visual reconstruction. In this study, we present an EEG-based visual reconstruction framew…

Generating, Reconstructing, and Representing Discrete and Continuous Data: Generalized Diffusion with Learnable Encoding-Decoding
Guangyi Liu, Yu Wang, Zeyu Feng, Qiyu Wu, Liping Tang, Yuan Gao, Zhen Li, Shuguang Cui, Julian McAuley, Eric P. Xing, Zichao Yang, Zhiting Hu
https://arxiv.org/abs/2402.19009

Generating, Reconstructing, and Representing Discrete and Continuous Data: Generalized Diffusion with Learnable Encoding-Decoding
The vast applications of deep generative models are anchored in three core capabilities -- generating new instances, reconstructing inputs, and learning compact representations -- across various data types, such as discrete text/protein sequences and continuous images. Existing model families, like Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), autoregressive models, and diffusion models, generally excel in specific capabilities and data types but fall short in others.…

Leveraging Diverse Modeling Contexts with Collaborating Learning for Neural Machine Translation
Yusheng Liao, Yanfeng Wang, Yu Wang
https://arxiv.org/abs/2402.18428

Leveraging Diverse Modeling Contexts with Collaborating Learning for Neural Machine Translation
Autoregressive (AR) and Non-autoregressive (NAR) models are two types of generative models for Neural Machine Translation (NMT). AR models predict tokens in a word-by-word manner and can effectively capture the distribution of real translations. NAR models predict tokens by extracting bidirectional contextual information which can improve the inference speed but they suffer from performance degradation. Previous works utilized AR models to enhance NAR models by reducing the training data's comp…

CollaFuse: Navigating Limited Resources and Privacy in Collaborative Generative AI
Domenique Zipperling, Simeon Allmendinger, Lukas Struppek, Niklas K\"uhl
https://arxiv.org/abs/2402.19105

CollaFuse: Navigating Limited Resources and Privacy in Collaborative Generative AI
In the landscape of generative artificial intelligence, diffusion-based models present challenges for socio-technical systems in data requirements and privacy. Traditional approaches like federated learning distribute the learning process but strain individual clients, especially with constrained resources (e.g., edge devices). In response to these challenges, we introduce CollaFuse, a novel framework inspired by split learning. Tailored for efficient and collaborative use of denoising diffusio…

This https://arxiv.org/abs/2402.12147 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
In this paper, we explore the challenges associated with establishing an end-to-end fact-checking pipeline in a real-world context, covering over 90 languages. Our real-world experimental benchmarks demonstrate that fine-tuning Transformer models specifically for fact-checking tasks, such as claim detection and veracity prediction, provide superior performance over large language models (LLMs) like GPT-4, GPT-3.5-Turbo, and Mistral-7b. However, we illustrate that LLMs excel in generative tasks …

This https://arxiv.org/abs/2310.05292 has been replaced.
link: https://scholar.google.com/scholar?q=a

How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging
Large Language Models (LLMs) now excel at generative skills and can create content at impeccable speeds. However, they are imperfect and still make various mistakes. In a Computer Science education context, as these models are widely recognized as "AI pair programmers," it becomes increasingly important to train students on evaluating and debugging the LLM-generated code. In this work, we introduce HypoCompass, a novel system to facilitate deliberate practice on debugging, where human novices p…

This https://arxiv.org/abs/2402.10466 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

Large Language Models as Zero-shot Dialogue State Tracker through Function Calling
Large language models (LLMs) are increasingly prevalent in conversational systems due to their advanced understanding and generative capabilities in general contexts. However, their effectiveness in task-oriented dialogues (TOD), which requires not only response generation but also effective dialogue state tracking (DST) within specific tasks and domains, remains less satisfying. In this work, we propose a novel approach FnCTOD for solving DST with LLMs through function calling. This method imp…

AI21 Labs launches Jamba, an AI model that integrates two architectures: transformer and Mamba, which is based on the Structured State Space model (Kyle Wiggers/TechCrunch)
https://techcrunch.com/2024/03/28/ai21-labs-new-text-g…

AI21 Labs' new AI model can handle more context than most | TechCrunch
Increasingly, the AI industry is moving toward generative AI models with longer contexts. But models with large context windows tend to be

This https://arxiv.org/abs/2402.14978 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csHC_…

AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation
The growing availability of generative AI technologies such as large language models (LLMs) has significant implications for creative work. This paper explores twofold aspects of integrating LLMs into the creative process - the divergence stage of idea generation, and the convergence stage of evaluation and selection of ideas. We devised a collaborative group-AI Brainwriting ideation framework, which incorporated an LLM as an enhancement into the group ideation process, and evaluated the idea g…

Analysis: Apple hired at least 36 AI experts from Google and has created a secretive European laboratory in Zurich, to develop new AI models and products (Michael Acton/Financial Times)
https://t.co/Iq7BirU6xu

Apple targets Google staff to build artificial intelligence team
iPhone maker also recruits generative AI experts into secretive Zurich lab as it prepares fightback against Big Tech rivals

This https://arxiv.org/abs/2404.07921 has been replaced.
initial toot: https://mastoxiv.page/@arXiv_csCL_…

AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs
As large language models (LLMs) become increasingly prevalent and integrated into autonomous systems, ensuring their safety is imperative. Despite significant strides toward safety alignment, recent work GCG~\citep{zou2023universal} proposes a discrete token optimization algorithm and selects the single suffix with the lowest loss to successfully jailbreak aligned LLMs. In this work, we first discuss the drawbacks of solely picking the suffix with the lowest loss during GCG optimization for jai…

Equivalence: An analysis of artists' roles with Image Generative AI from Conceptual Art perspective through an interactive installation design practice
Yixuan Li, Dan C. Baciu, Marcos Novak, George Legrady
https://arxiv.org/abs/2404.18385

Equivalence: An analysis of artists' roles with Image Generative AI from Conceptual Art perspective through an interactive installation design practice
Over the past year, the emergence of advanced text-to-image Generative AI models has significantly impacted the art world, challenging traditional notions of creativity and the role of artists. This study explores how artists interact with these technologies, using a 5P model (Purpose, People, Process, Product, and Press) based on Rhodes' creativity framework to compare the artistic processes behind Conceptual Art and Image Generative AI. To exemplify this framework, a practical case study titl…